Scribble-Supervised Video Object Segmentation

Abstract

Recently, video object segmentation has received great attention in the computer vision community. Most of the existing methods heavily rely on the pixel-wise human annotations, which are expensive and time-consuming to obtain.

To tackle this problem, we make an early attempt to achieve video object segmentation with scribble-level supervision, which can alleviate large amounts of human labor for collecting the manual annotation. However, using conventional network architectures and learning objective functions under this scenario cannot work well as the supervision information is highly sparse and incomplete.

Methodology

To address this issue, this paper introduces two novel elements to learn the video object segmentation model:

1. Scribble Attention Module: Captures more accurate context information and learns an effective attention map to enhance the contrast between foreground and background.

2. Scribble-Supervised Loss: Can optimize the unlabeled pixels and dynamically correct inaccurate segmented areas during the training stage.

These two components work together to enable effective learning from sparse scribble annotations, making it possible to train high-quality video object segmentation models without requiring dense pixel-wise labels.

Experimental Results

To evaluate the proposed method, we implement experiments on two video object segmentation benchmark datasets:

• YouTube-VOS (YouTube-Video Object Segmentation)

• DAVIS-2017 (Densely Annotated Video Segmentation)

We first generate the scribble annotations from the original per-pixel annotations. Then, we train our model and compare its test performance with the baseline models and other existing works.

Extensive experiments demonstrate that the proposed method can work effectively and approach to the methods requiring the dense per-pixel annotations, while using significantly less supervision.

This demonstrates that scribble-level supervision is a viable and efficient alternative to pixel-wise annotations for video object segmentation tasks.